Neural Networks Compression for Language Modeling

In this paper, we consider several compression techniques for the language modeling problem based on recurrent neural networks (RNNs). It is known that conventional RNNs, e.g, LSTM-based networks in language modeling, are characterized with either high space complexity or substantial inference time. This problem is especially crucial for mobile applications, in which the constant interaction with the remote server is inappropriate. By using the Penn Treebank (PTB) dataset we compare pruning, quantization, low-rank factorization, tensor train decomposition for LSTM networks in terms of model size and suitability for fast inference.Comment: Keywords: LSTM, RNN, language modeling, low-rank factorization, pruning, quantization. Published by Springer in the LNCS series, 7th International Conference on Pattern Recognition and Machine Intelligence, 201

Averages of Fourier coefficients of Siegel modular forms and representation of binary quadratic forms by quadratic forms in four variables

Let $-d$ be a a negative discriminant and let $T$ vary over a set of representatives of the integral equivalence classes of integral binary quadratic forms of discriminant $-d$. We prove an asymptotic formula for $d \to \infty$ for the average over $T$ of the number of representations of $T$ by an integral positive definite quaternary quadratic form and obtain results on averages of Fourier coefficients of linear combinations of Siegel theta series. We also find an asymptotic bound from below on the number of binary forms of fixed discriminant $-d$ which are represented by a given quaternary form. In particular, we can show that for growing $d$ a positive proportion of the binary quadratic forms of discriminant $-d$ is represented by the given quaternary quadratic form.Comment: v5: Some typos correcte

Prodsimplicial-Neighborly Polytopes

Simultaneously generalizing both neighborly and neighborly cubical polytopes, we introduce PSN polytopes: their k-skeleton is combinatorially equivalent to that of a product of r simplices. We construct PSN polytopes by three different methods, the most versatile of which is an extension of Sanyal and Ziegler's "projecting deformed products" construction to products of arbitrary simple polytopes. For general r and k, the lowest dimension we achieve is 2k+r+1. Using topological obstructions similar to those introduced by Sanyal to bound the number of vertices of Minkowski sums, we show that this dimension is minimal if we additionally require that the PSN polytope is obtained as a projection of a polytope that is combinatorially equivalent to the product of r simplices, when the dimensions of these simplices are all large compared to k.Comment: 28 pages, 9 figures; minor correction

Form Factors and Wave Functions of Vector Mesons in Holographic QCD

Within the framework of a holographic dual model of QCD, we develop a formalism for calculating form factors of vector mesons. We show that the holographic bound states can be described not only in terms of eigenfunctions of the equation of motion, but also in terms of conjugate wave functions that are close analogues of quantum-mechanical bound state wave functions. We derive a generalized VMD representation for form factors, and find a very specific VMD pattern, in which form factors are essentially given by contributions due to the first two bound states in the Q^2-channel. We calculate electric radius of the rho-meson, finding the value _C = 0.53 fm^2.Comment: 7 pages, RevTex. References were added, some modifications in the text were mad

On complex surfaces diffeomorphic to rational surfaces

In this paper we prove that no complex surface of general type is diffeomorphic to a rational surface, thereby completing the smooth classification of rational surfaces and the proof of the Van de Ven conjecture on the smooth invariance of Kodaira dimension.Comment: 34 pages, AMS-Te

Volumes of polytopes in spaces of constant curvature

We overview the volume calculations for polyhedra in Euclidean, spherical and hyperbolic spaces. We prove the Sforza formula for the volume of an arbitrary tetrahedron in $H^3$ and $S^3$. We also present some results, which provide a solution for Seidel problem on the volume of non-Euclidean tetrahedron. Finally, we consider a convex hyperbolic quadrilateral inscribed in a circle, horocycle or one branch of equidistant curve. This is a natural hyperbolic analog of the cyclic quadrilateral in the Euclidean plane. We find a few versions of the Brahmagupta formula for the area of such quadrilateral. We also present a formula for the area of a hyperbolic trapezoid.Comment: 22 pages, 9 figures, 58 reference

Using exchange structure analysis to explore argument in text-based computer conferences

Computer conferencing provides a new site for students to develop and rehearse argumentation skills, but much remains to be learnt about how to encourage and support students in this environment. Asynchronous text-based discussion differs in significant ways from face-to-face discussion, creating a need for specially designed schemes for analysis. This paper discusses some of the problems of analysing asynchronous argumentation, and puts forward an analytical framework based on exchange structure analysis, which brings a linguistic perspective to bear on the interaction. Key features of the framework are attention to both interactive and ideational aspects of the discussion, and the ability to track the dynamic construction of argument content. The paper outlines the framework itself, and discusses some of the findings afforded by this type of analysis, and its limitations

Small doubling in groups

Let A be a subset of a group G = (G,.). We will survey the theory of sets A with the property that |A.A| <= K|A|, where A.A = {a_1 a_2 : a_1, a_2 in A}. The case G = (Z,+) is the famous Freiman--Ruzsa theorem.Comment: 23 pages, survey article submitted to Proceedings of the Erdos Centenary conferenc

Bilingually motivated word segmentation for statistical machine translation

We introduce a bilingually motivated word segmentation approach to languages where word boundaries are not orthographically marked, with application to Phrase-Based Statistical Machine Translation (PB-SMT). Our approach is motivated from the insight that PB-SMT systems can be improved by optimizing the input representation to reduce the predictive power of translation models. We firstly present an approach to optimize the existing segmentation of both source and target languages for PB-SMT and demonstrate the effectiveness of this approach using a Chinese–English MT task, that is, to measure the influence of the segmentation on the performance of PB-SMT systems. We report a 5.44% relative increase in Bleu score and a consistent increase according to other metrics. We then generalize this method for Chinese word segmentation without relying on any segmenters and show that using our segmentation PB-SMT can achieve more consistent state-of-the-art performance across two domains. There are two main advantages of our approach. First of all, it is adapted to the specific translation task at hand by taking the corresponding source (target) language into account. Second, this approach does not rely on manually segmented training data so that it can be automatically adapted for different domains